Image Processing for Art Investigation

Recent advances in digital image acquisition methods and the wide range of imaging modalities currently available have triggered museums to digitize their painting collections. Not only is this crucial for archival or dissemination purposes but it also enabled the digital analysis of the painting through its digital image counterpart. It also set in motion a cross-disciplinary collaboration between image analysis specialists, mathematicians, statisticians and art historians that have the common goal to develop algorithms and build a digital toolbox in support of art scholarship. Computer processing of digital images of paintings has become a fast growing and challenging field of research during the last few years. Our contribution to this research domain consists of a set of tools that are based on dimensionality reduction methods, sparse representations and dictionary learning techniques. These tools are used to assist in art related matters such as restoration, conservation, art history, material and structure characterization, authentication, dating and even style analysis. Since paintings are complex structures the analysis of all pictorial layers and the support requires a multimodal set of high-resolution image acquisitions. The presented research can broadly be subdivided into three main fields. The first one is the digital enhancement of painting acquisitions in order to assist the art specialist in his professional assessment of the painting. The second main field of research is the automated detection of cracks within the Ghent Altarpiece, which is meant to help in the delicate matter of the conservation of this exceptional masterpiece but also as guidance during its current campaign of restoration. The last field consists of a set of methods that can be deployed in art forensics. These methods consist of the characterization of canvas, the analysis of multispectral imagery of a painting and even the objective quantification of the style of a particular artist.

Efficiency analysis methodology of FPGAs based on lost frequencies, area and cycles

We propose a methodology to study and to quantify efficiency and the impact of overheads on runtime performance. Most work on High-Performance Computing (HPC) for FPGAs only studies runtime performance or cost, while we are interested in how far we are from peak performance and, more importantly, why. The efficiency of runtime performance is defined with respect to the ideal computational runtime in absence of inefficiencies. The analysis of the difference between actual and ideal runtime reveals the overheads and bottlenecks. A formal approach is proposed to decompose the efficiency into three components: frequency, area and cycles. After quantification of the efficiencies, a detailed analysis has to reveal the reasons for the lost frequencies, lost area and lost cycles. We propose a taxonomy of possible causes and practical methods to identify and quantify the overheads. The proposed methodology is applied on a number of use cases to illustrate the methodology. We show the interaction between the three components of efficiency and show how bottlenecks are revealed

The Effect of Space-filling Curves on the Efficiency of Hand Gesture Recognition Based on sEMG Signals

Over the past few years, Deep learning (DL) has revolutionized the field of data analysis. Not only are the algorithmic paradigms changed, but also the performance in various classification and prediction tasks has been significantly improved with respect to the state-of-the-art, especially in the area of computer vision. The progress made in computer vision has produced a spillover in many other domains, such as biomedical engineering. Some recent works are directed towards surface electromyography (sEMG) based hand gesture recognition, often addressed as an image classification problem and solved using tools such as Convolutional Neural Networks (CNN). This paper extends our previous work on the application of the Hilbert space-filling curve for the generation of image representations from multi-electrode sEMG signals, by investigating how the Hilbert curve compares to the Peano- and Z-order space-filling curves. The proposed space-filling mapping methods are evaluated on a variety of network architectures and in some cases yield a classification improvement of at least 3%, when used to structure the inputs before feeding them into the original network architectures

Multi-modal dictionary learning for image separation with application in art investigation

In support of art investigation, we propose a new source separation method that unmixes a single X-ray scan acquired from double-sided paintings. In this problem, the X-ray signals to be separated have similar morphological characteristics, which brings previous source separation methods to their limits. Our solution is to use photographs taken from the front and back-side of the panel to drive the separation process. The crux of our approach relies on the coupling of the two imaging modalities (photographs and X-rays) using a novel coupled dictionary learning framework able to capture both common and disparate features across the modalities using parsimonious representations; the common component models features shared by the multi-modal images, whereas the innovation component captures modality-specific information. As such, our model enables the formulation of appropriately regularized convex optimization procedures that lead to the accurate separation of the X-rays. Our dictionary learning framework can be tailored both to a single- and a multi-scale framework, with the latter leading to a significant performance improvement. Moreover, to improve further on the visual quality of the separated images, we propose to train coupled dictionaries that ignore certain parts of the painting corresponding to craquelure. Experimentation on synthetic and real data - taken from digital acquisition of the Ghent Altarpiece (1432) - confirms the superiority of our method against the state-of-the-art morphological component analysis technique that uses either fixed or trained dictionaries to perform image separation.Comment: submitted to IEEE Transactions on Images Processin

X-ray image separation via coupled dictionary learning

In support of art investigation, we propose a new source sepa- ration method that unmixes a single X-ray scan acquired from double-sided paintings. Unlike prior source separation meth- ods, which are based on statistical or structural incoherence of the sources, we use visual images taken from the front- and back-side of the panel to drive the separation process. The coupling of the two imaging modalities is achieved via a new multi-scale dictionary learning method. Experimental results demonstrate that our method succeeds in the discrimination of the sources, while state-of-the-art methods fail to do so.Comment: To be presented at the IEEE International Conference on Image Processing (ICIP), 201

Performance and toolchain of a combined GPU/FPGA desktop

Low-power, high-performance computing nowadays relies on accelerator cards to speed up the calculations. Combining the power of GPUs with the flexibility of FPGAs enlarges the scope of problems that can be accelerated. We describe the performance analysis of a desktop equipped with a GPU Tesla 2050 and an FPGA Virtex- 6 LX 240T. The balance between the I/O and the raw peak performance is analyzed using the roofline model. A well-tuned accelerator- based codesign, identifying the parallelism, the computation and data patterns of different classes of algorithms, will enable to maximize the performance of the combined GPU/FPGA system

Study of combining GPU/FPGA accelerators for high-performance computing

This contribution presents the performance modeling of a super desktop with GPU and FPGA accelerators, using OpenCL for the GPU and a high-level synthesis compiler for the FPGAs. The performance model is used to evaluate the different high-level synthesis optimizations, taking into account the resource usage, and to compare the compute power of the FPGA with the GP

Image Processing for Art Investigation

Advisors: Ann Dooms, Ingrid Daubechies. Date and location of PhD thesis defense: 13 October 2014, Vrije Universiteit BrusselRecent advances in digital image acquisition methods and the wide range of imaging modalities currently available have triggered museums to digitize their painting collections. Not only is this crucial for archival or dissemination purposes but it also enabled the digital analysis of the painting through its digital image counterpart. It also set in motion a cross-disciplinary collaboration between image analysis specialists, mathematicians, statisticians and art historians that have the common goal to develop algorithms and build a digital toolbox in support of art scholarship. Computer processing of digital images of paintings has become a fast growing and challenging field of research during the last few years. Our contribution to this research domain consists of a set of tools that are based on dimensionality reduction methods, sparse representations and dictionary learning techniques. These tools are used to assist in art related matters such as restoration, conservation, art history, material and structure characterization, authentication, dating and even style analysis. Since paintings are complex structures the analysis of all pictorial layers and the support requires a multimodal set of high-resolution image acquisitions. The presented research can broadly be subdivided into three main fields. The first one is the digital enhancement of painting acquisitions in order to assist the art specialist in his professional assessment of the painting. The second main field of research is the automated detection of cracks within the Ghent Altarpiece, which is meant to help in the delicate matter of the conservation of this exceptional masterpiece but also as guidance during its current campaign of restoration. The last field consists of a set of methods that can be deployed in art forensics. These methods consist of the characterization of canvas, the analysis of multispectral imagery of a painting and even the objective quantification of the style of a particular artist

Spatiogram features to characterize pearls in paintings

Objective characterization of jewels in paintings, especially pearls, has been a long lasting challenge for art historians. The way an artist painted pearls reflects his ability to observing nature and his knowledge of contemporary optical theory. Moreover, the painterly execution may also be considered as an individual characteristic useful in distinguishing hands. In this work, we propose a set of image analysis techniques to analyze and measure spatial characteristics of the digital images of pearls, all relying on the so called spatiogram image representation. Our experimental results demonstrate good correlation between the new metrics and the visually observed image features, and also capture the degree of realism of the visual appearance in the painting. In that sense, these results set the basis in creating a practical tool for art historical attribution and give strong motivation for further investigations in this direction